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An  Evaluation  of  Aviation  Maintenance 
Fatigue  Countermeasures  Training 


Aviation  maintenance  is  a  24/7  operation  requiring  Avia¬ 
tion  Maintenance  Technicians  (AMTs)  to  work  extended 
hours,  nights,  and  rotating  shifts  to  meet  demanding  flight 
schedules.  Under  the  Code  of  Federal  Regulations  (Title 
14  CFR  §121.377),  AMTs  are  allowed  to  work  24  hours 
per  day  until  their  next  day  off.  Maintenance  organizations 
have  been  slow  to  implement  fatigue  risk  management 
policies  despite  fatigue-related  catastrophic  events  (e.g., 
Valujet Flight 592 with  llOfatalities  (AAR-97-06)).Tothe 
contrary,  some  maintenance  organizations  have  instituted 
a  double  standard  by  imposing  strict  “no  nesting”'  and  “no 
napping”  policies  on  AMTs,  while  permitting  them  to  work 
or  exceed  three  concecutive  16-hour  shifts.  Such  policies 
are  counter  to  personal,  organizational,  and  public  safety 
and  foster  a  culture  where  it  is  acceptable  (and  expected)  to 
work  when  fatigued.  This  situation  is  partially  driven  by  the 
AMT’s  need  to  earn  overtime  wages  and  the  organization’s 
need  to  have  employees  on  duty  for  as  long  as  necessary. 


that  accumulated  to  levels  comparable  to  that  found  after 
severe  acute  total  sleep  deprivation.  The  Banks  and  Dinges 
study  resolves  any  question  of  whether  or  not  the  flve  hours 
of  sleep  that  AMTs  typically  get  is  adequate. 

Fatigue  countermeasures  training  has  been  used  for 
more  than  20  years  across  industries  (railroad,  trucking, 
and  water  transport)  with  24/7  operations,  to  mitigate 
on-the-job  fatigue  risks  (Nicholson  &  Stone,  1987).  The 
training  can  be  beneficial  to  both  the  individual  and  the 
organization.  Individuals  who  replace  bad  sleep  and  health 
habits  with  good  ones  benefit  from  improved  sleep  quality 
and  quantity,  while  the  organization  benefits  from  improve¬ 
ments  in  performance  and  fewer  safety-reducing  turnovers, 
absenteeism,  and  morale  issues  (Kerin  &  Aguirre,  2005). 

The  Federal  Aviation  Administration  (FAA)  is  commit¬ 
ted  to  promoting  aviation  safety  and  has  tasked  the  Civil 
Aerospace  Medical  Institute  (CAMI)  to  research  and  de¬ 
velop  the  tools  necessary  for  a  practical  and  useful  fatigue 


Table  1.  Content  covered  in  the  training 

Sleep  Basics 

Fatigue  Basics 

Fatigue  Countermeasures 

Sleep  Proeess 

Fatigue  Hazards 

Work  Breaks 

Circadian  Rhythm 

Causes  of  Fatigue 

Napping 

Sleep  Disorders 

How  to  Assess  Fatigue 

Sleep  Routine 

Sleep  Debt 

Fatigue  Symptoms 

Sleeping  Environment 

No  one  is  immune  to  on-the-job  fatigue  risks,  especially 
those  working  shiftwork^  (Miller,  2012).  For  the  past  decade, 
AMTs  have  been  working  longer  hours  to  compensate  for 
the  wage  reductions  since  the  911 1  terrorist  attacks  in  200 1 . 
Long  work  hours,  especially  weekly  schedules  at  the  60 
hour  or  above  mark,  can  lead  to  problems  in  certain  areas 
of  health  and  safety  (Allen,  Slavin,  &  Bunn,  2007).  In  a 
2001  study  examining  AMT  sleep  and  rest  periods,  results 
revealed  AMTs  were  only  getting  about  flve  hours  of  sleep, 
on  average,  per  night  (Johnson,  Mason,  Hall,  &  Watson, 
200 H).  More  recently.  Banks  and  Dinges  (2007)  found 
chronic  sleep  restriction  below  seven  hours  per  night  to  be 
associated  with  significant  daytime  cognitive  dysfunction 

'  Building  a  bed  pallet  at  the  work-site  in  order  to  sleep  between  shifts. 
^  Shiftwork  refers  to  any  of  the  following:  permanent  evening  or  night 
shift,  rotating  schedule  with  or  without  nights,  split  shift,  extended 
daytime  hours  (start  before  7:00  am  or  end  after  5:00  pm)  and  extended 
duty  hours  (work  period  of  12+  hours). 

^  This  study  was  completed  before  the  airline  economic  crisis  triggered 
by  the  9111  terrorist  attacks  in  2001. 


risk  management  system  (FRMS).  Developing  fatigue 
countermeasures  training  tailored  to  aviation  maintenance 
is  an  obvious  starting  place.  The  remainder  of  this  report 
describes  the  training  development  process  and  presents  the 
outcomes  of  the  evaluation  conducted  on  initial  delivery  of 
the  training  in  a  classroom  setting  to  maintenance  personnel 
from  a  major  airline. 

aviation  maintenance  fatigue 
countermeasures  training 

Training  Development 

The  development  effort  capitalized  on  existing  training 
and  used  maintenance  subject  matter  experts  (SMEs)  to 
identify  specific  work  requirements  of  AMTs.  Existing 
fatigue  training  programs  were  identified  and  reviewed 
for  purposes  of  creating  a  course  outline  and  objectives. 
The  outline  covered  three  main  topics:  sleep  basics,  fatigue 
basics,  and  fatigue  countermeasures.  Table  1  contains  the 
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subject  matter  covered  under  each  topic.  The  content  was 
reviewed  by  two  SMEs  for  deficiencies,  excesses,  and  ac¬ 
curacy  (Hauck,  Avers,  Banks,  &  Blackwell,  2011).  The 
final  step  was  to  develop  PowerPoint®  slides  covering  the 
content  and  relevant  multimedia  (i.e.,  audio,  video,  and 
other  visual  media). 

Training  objectives  were:  (a)  remember  symptoms  of 
fatigue  and  fatigue  countermeasures,  (b)  recognize  the 
importance  of  managing  fatigue  risk,  and  (c)  incorporate 
practical  recommendations  for  fatigue  prevention  and  man¬ 
agement  into  one’s  daily  routine.  Training  outcome  measures 
used  to  determine  if  the  obj  ectives  were  met  included:  gains 
in  knowledge,  increased  awareness  of  fatigue  risk  and  the 
importance  of  managing  fatigue,  and  an  increase  in  self- 
report  of  fatigue-related  positive  behaviors  and  reduction 
of  negative  behaviors  at  home  and  work. 

Training  Delivery 

A  major  airline  agreed  to  participate  in  the  initial  imple¬ 
mentation  of  this  evaluation.  The  course  was  delivered  in  a 
classroom  at  their  facility  with  33  maintenance  employees 
attending  on  a  voluntary  basis.  A  CAMI  research  team 
member  with  extensive  experience  delivering  classroom 
instruction  presented  the  course.  The  course  was  delivered 
in  a  3-hour  session,  with  15-minute  breaks  between  sec¬ 
tions.  The  instructor  used  PowerPoint®  slides  projected  on 
a  large  screen  at  the  front  of  the  classroom  to  present  the 
training.  At  the  conclusion  of  each  section,  the  instructor 
led  a  group  discussion  and  then  reviewed  the  content  with 
the  class  before  conducting  a  check  on  understanding  and 
knowledge  with  feedback. 

Training  Evaluation 

The  evaluation  was  conducted  to  validate  the  training  in 
preparation  for  making  it  available  for  public  access  online, 
as  self-paced  instruction.'*  The  delivered  training  was  evalu¬ 
ated  on  four  levels:  learning  (positive  change  in  knowledge 
of  and  attitude  toward  countering  fatigue),  behavior 
modification  (change  in  fatigue  management  behaviors  at 
home  and  work),  transfer  (application  of  what  was  taught 
at  home  and  work) ,  and  reaction  to  the  training  experience. 

METHOD 

A  quasi-experimentaP  one-group  pretest-posttest  design 
with  a  follow-up  assessment  was  used  to  evaluate  the  training. 


*  https:/ /hfskyway.faa.gov/hfskyway/ fatiguehome.aspx. 

^  There  were  insufficient  resources  to  support  a  control  group,  and  it 
was  not  feasible  to  randomly  assign  AMTs  in  the  airline’s  workforce. 


Training  Participants 

Evaluation  data  were  collected  from  42  maintenance 
personnel  working  for  a  major  airline.  However,  for  various 
reasons,  not  all  42  attended  training.  Of  the  33  who  did 
attend  training,  24  were  able  to  participate  in  both  of  the 
first  two  data  collections,  which  were  conducted  prior  to 
and  at  the  end  of  the  training  session.  However,  only  14  of 
them  chose  to  participate  in  the  follow-up  data  collection 
several  weeks  later.  Demographic  information  from  24  at¬ 
tendees  whose  evaluation  data  were  used  in  the  analysis  is 
reported  at  the  beginning  of  the  Results  section. 

Procedures 

A  week  prior  to  the  training,  potential  attendees  were 
asked  to  complete  either  an  online  or  paper  version  of  the 
pre-measure  that  was  available  at  the  airline’s  training  of¬ 
fice.  At  the  end  of  the  training,  a  paper  post-measure  was 
administered.  After  handing  in  the  post-measure,  each 
attendee  received  a  packet  of  information  to  take  home 
(e.g.,  copy  of  the  training  slides  and  handouts  on  key 
topics  covered  in  training).  They  were  also  reminded  that 
they  would  be  contacted  in  four  to  six  weeks  to  complete 
a  questionnaire,  either  online  or  paper.  Six  weeks  follow¬ 
ing  training,  attendees  were  contacted  via  email  or  mail  to 
complete  the  final  measure. 

Measures 

All  measures  contained  items  to  assess  knowledge  of  sleep 
and  fatigue  basics  and  fatigue  risk**  and  attitudes  toward 
managing  fatigue  risk.  Table  2  contains  an  overview  of  the 
measures  (copies  of  instruments  available  upon  request). 

Before  conducting  statistical  comparisons  of  the  evalua¬ 
tion  data,  knowledge  tests  were  scored  and  percent  correct 
computed;  response  values  (5-point  scale)  for  the  10-item 
Fatigue  Assessment  Scale  (FAS)  were  summed  to  produce 
a  total  score^  (Michielsen,  De  Vries,  &  Van  Heck,  2003), 
and  multi-item  categories  were  created  for  the  Attitude  As¬ 
sessment  and-Fatigue  Management  Behavioral  Inventory. 
Categories  created  for  the  Attitude  Assessment  items  were: 
awareness  of  the  importance  of  managing  fatigue,  self-efficacy 
applying  fatigue  countermeasures,  and  intent  to  modify  be¬ 
havior.  Categories  created  for  the  Behavioral  Inventory  items 
aligned  v^'v^good habits  (sleep  routine,  health  &  fitness,  and 
work-life)  and  bad  habits  regarding  fatigue  management. 

In  addition,  individual  items  from  the  remaining  assess¬ 
ment  instruments,  as  direct  measures  of  training  effective¬ 
ness,  were  targeted  for  comparison  across  data  collections. 


Mlthough  the  three  administrations  contained  identical  knowledge 
test  items,  sufficient  time  elapsed  between  administrations  to  mitigate 
an  effect  of  practice,  with  the  exception  of  a  few  attendees  who  may 
have  completed  a  paper  version  of  the  pre-measure  the  day  of  training. 
^  Two  FAS  items  were  reversed  scored — higher  values  represent  more 
fatigue. 
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Table  2.  Number  of  assessment  instruments  and  items  across  administrations 


Pre- 

Post- 

Follow-up 

measure 

measure 

measure 

Prior  Fatigue  Education/Training 

5 

Demographics 

17 

Recall  Knowledge 

25 

25 

25 

Recognition  Knowledge^ 

15 

15 

15 

Attitude  Assessment 

14 

14 

14 

Fatigue  Assessment  Scale 

10 

10 

Fatigue  Management  Behavioral  Inventory 

44 

44 

Physical  Symptoms  Self-report 

19 

19 

Sleep  Habits 

6 

6 

Reaction  to  the  Training  Experience 

9 

Behavioral  Change  Self-report 

4 

4 

Nine  recall  items  (short  answer)  and  five  recognition  items  (multiple-choice)  were  excluded  in  computing  the 
knowledge  scores,  due  to  lack  of  relevance  to  covered  content. 


The  targeted  items  included:  the  extent  to  which  tiredness 
or  fatigue  and  trouble  sleeping  were  experienced  in  the  last 
30  days  from  the  Physical  Symptoms  Self-report  (5-point 
scale,  the  items  were  reversed  scored  so  high  values  indicated 
a  lesser  extent);  adequacy  of  the  amount  of  sleep  on  work 
and  off  days  from  the  Sleep  Habits  Self-report  (5-point 
scale  with  high  values  indicating  more  sleep);  and  changes 
made  to  manage  fatigue  at  home  and  work  (yes  or  no;  if 
yes,  how),  which  constituted  the  four  Behavioral  Change 
Self-Report  items. 


will  persist  (e.g. ,  follow-up  test  scores  would  be  higher  than 
pre-training  scores),  but  may  degrade  (e.g.,  post- training 
test  scores  would  be  higher  than  follow-up  scores).  To  as¬ 
sess  training  transfer,  behavioral  follow-up  measures  and 
pre-measures  were  compared,  with  an  expected  increase  in 
the  frequency  of  use  of  countermeasures  at  home  and  work, 
and  an  expected  decrease  in  the  frequency  of  self-reported 
poor  sleeping,  eating,  and  exercise  habits. 

RESULTS 


Analysis  Plan 

The  purpose  of  the  evaluation  was  twofold:  (a)  examine 
the  potential  of  the  course  content  in  meeting  the  training 
objectives  and  (b)  identify  needed  improvements  before 
converting  the  content  to  online  training. 

Needed  training  improvements  were  extracted  from  a 
content  analysis  of  open-ended,  text-entry  items  on  the 
post-measure  asking  which  topics  were  most  and  least 
useful  and  what  could  be  changed  to  improve  the  training. 

Statistics  were  used  to  describe  and  test®  for  differences 
in  average  scores  and  ratings  and  response  frequencies  of 
self-report  data.  The  following  pairwise-comparisons  were 
made  to  determine  if  training  objectives  were  met.  To  assess 
learning,  pre-  and  post-measures  were  compared,  with  the 
expectation  that  training  has  an  immediate  positive  impact 
on  knowledge  scores  and  attitudes  regarding  the  importance 
of  and  motivation  and  commitment  to  managing  fatigue. 
To  assess  knowledge  retention  and  persistence  of  positive 
attitudes  toward  fatigue  management,  follow-up  measures 
were  compared  to  pre-  and  post-measures,  with  the  expec¬ 
tation  that  knowledge  gains  and  positive  attitude  change 

®  Two-tailed  tests  were  used  with  the  conventional  0.05  p  value. 


Table  3  displays  the  characteristics  of  participants  who 
attended  the  fatigue  countermeasures  training  and  com¬ 
pleted  some,  if  not  all,  of  the  measures  being  compared. 

Twenty-four  attendees  completed  items  on  both  of  the 
pre-  and  post-  measures.  A  subset  of  the  24  individual  also 
completed  items  on  the  follow-up  measure.  Thus,  the  num¬ 
ber  of  participants  («)  and  average  scores  and  self-ratings 
differed,  as  indicated  for  pairwise  comparisons  conducted 
across  the  three  data  collection  events. 

The  following  are  the  evaluation  results  of  training  im¬ 
pacts  on  learning,  retention,  transfer,  and  reaction  to  the 
training  experience;  changes  in  self-reported  fatigue;  and 
needed  training  improvements. 

Learning 

Learning  was  measured  as  the  immediate  impact  of  train¬ 
ing  on  attendee  knowledge  and  attitude.  Gain  in  knowl¬ 
edge  was  computed  as  post-  minus  pre-training  scores  on 
short- answer  and  multiple-choice  tests.  Change  in  attitude 
was  computed  in  the  same  manner  for  the  awareness  and 
self-efficacy  items  from  the  Attitude  Assessment. 
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Table  3.  Characteristics  of  24  attendees  who  provided  data  for  the  evaluation 

Type  of  Mx 
performed^ 
(%) 

Avg 

Age 

(range) 

Avg  years 
of  job 
experience 
(range) 

Avg 

weekly 

work 

hrs 

(range) 

Avg 
weekly 
OT  hrs 
(range) 

Typical  work 
schedule^ 

(%) 

Received 

prior 

fatigue 

training/ 

info 

Prior 
training/ 
info  was 
helpful 
(n=6) 

17. 5 - Line 

17.5- 

Component 

13.0-Fleavy 

52.0-Other^ 

50.4 

(41-61) 

29.0 

(13.0-42.8) 

43.0 

(10-70) 

2.8 

(0-16) 

8.6-Rotating 
with  afternoon 
and  nights 

9 1.3 -Day/ 
extended  day 

25% 

33.3% 

Mx  -  maintenance;  Avg  -  average  (mean);  hrs  -  hours;  OT  -  overtime 
missing  data  from  1  attendee 
^Mx  management  and  training  personnel 


Figure  1.  Immediate  impact  of  training  on 
knowledge  test  scores  (mean)  (n=23) 


Gain  in  knowledge.  There  was  an  immediate  impact  of 
training  on  attendee  fatigue-related  knowledge.  Average  test 
scores  increased  by  50%,  going  from  50.0%  correct  before 
training  to  75.2%  correct  at  the  end  of  training  (see  Figure 
1).  The  gain  was  statistically  significant  =  25.2,  t^^  = 
8.42,  pvalue  <  .0001). 

Shift  in  attitude.  Although  shifts  in  attendee  awareness 
of  the  importance  of  managing  fatigue  and  self-efficacy 
trended  in  the  positive  direction  (see  Figure  2),  training 
only  had  an  immediate  positive  impact  on  awareness  of 
importance  not  to  be  fatigued  at  work  (Mj,|j,=.33,  t23=2.56, 
pvalue  <.02)  and  commitment  to  manage  fatigue  = 
.46,  t^j  =2.14,  pvalue  <.05). 


Retention 

Retention  reflects  a  relatively  stable  gain  in  knowledge 
and  change  in  attitude.  It  was  measured  as  differences  be¬ 
tween  follow-up  and  post-training.  Comparisons  were  also 
made  between  follow-up  and  pre-training  for  test  scores 
and  ratings  on  the  Attitude  Assessment. 

Knowledge.  Attendees  retained  what  they  learned.  Figure 
3  shows  average  follow-up  knowledge  test  scores  compared 
to  post-  and  pre-training  scores.  A  slight  dip  in  scores 
(M3.g.=  -6.4,  NS)  occurred  on  the  6-week  follow-up  test. 
Of  importance  was  that  the  scores  remained  significantly 
higher  (by  40%)  on  the  follow-up  test  compared  to  before 
training  (Mj.^=19.6,  tj3=3.86,  pvalue  <.002). 
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□  Pre-T  raining  ■  Post-T  raining 


Awareness  of  Importance 

•How  important  do  you  think  it  is  to 
not  be  fatigued  while  on  duty  in  your 
line  of  work? 

To  what  extent  do  you  feel  that 
managing  fatigue  is  important? 

•How  committed  are  you  to 
managing  fatigue? 

How  motivated  are  you  to  avoid 
feeling  fatigued? 


< . Less  More . > 

12  3  4 


Tel 


3.8 


5 


Self-Efficacy 

To  what  extent  do  you  think  you  can 
successfully  manage  fatigue? 

How  confident  are  you  that  you  will 
be  able  to  apply  the  strategies 
learned  in  training? 

To  what  extent  do  you  think  that  you 
can  apply  strategies  to  manage 
fatigue  at  home? 

To  what  extent  do  you  think  that  you 
can  apply  strategies  to  manage 
fatigue  on  the  job? 


Figure  2.  Immediate  impact  of  training  on  attitudes  (mean  rating) 
(n=24;  *signifies  statisticaiiy  significant  shift) 


Pre-Training  Post-Training  Follow-up 


Figure  3.  Knowiedge  test  scores  (mean)  across 
administrations 
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■  Post-Training  ■Follow-Up 


Awareness  of  Importance 
How  important  do  you  think  it  is  to 
not  be  fatigued  while  on  duty  in  your 
line  of  work? 

To  what  extent  do  you  feel  that 
managing  fatigue  is  important? 

•How  committed  are  you  to 
managing  fatigue?  (n=15) 

•How  motivated  are  you  to  avoid 
feeling  fatigued? 


< — Less  More — > 

12  3  4  5 


Self-Efficacy 

•T o  what  extent  do  you  think  you 
can  successfully  manage  fatigue? 

How  confident  are  you  that  you  will 
be  able  to  apply  the  strategies 
learned  in  training? 

•To  what  extent  do  you  think  that 
you  can  apply  strategies  to  manage 
fatigue  at  home? 

To  what  extent  do  you  think  that  you 
can  apply  strategies  to  manage 
fatigue  on  the  job? 


< — Less  More — > 


3. 


mJ 


3. 


Figure  4.  Retention  of  attitudes  between  training  and  foiiow-up  (mean) 
(n=16;  ‘signifies  statisticaiiy  significant  shift) 


Attitudes.  Figure  4  shows  the  downward  trending  of 
awareness,  self-efficacy,  and  intention  toward  fatigue  man¬ 
agement  six  weeks  following  training.  The  positive  shift  in 
awareness  of  importance  and  commitment  to  managing 
fatigue  realized  immediately  after  training  did  not  persist  six 
weeks  later.  In  fact,  the  decrease  in  attendee  commitment 
was  statistically  significant  -.56,  t^^  =  -2.52,  pvalue 

<.02),  and  their  motivation  to  avoid  fatigue  also  dipped 
-.40,  tj^  =  -2.45,  pvalue  <.03).  As  well,  with  time  to 
potentially  use  the  countermeasures  presented  in  training, 
there  was  a  statistically  significant  loss  in  the  sense  of  being 
able  to  successfully  manage  fatigue  in  general  -.50, 

tj^  =  -2.74,  pvalue  <.02)  and,  specifically  at  home 
-.44,  tj^  =  -3.42,  pvalue  <.004).  Overall,  none  of  the  shifts 
in  attitude  fell  below  the  mid-points  of  the  various  scales 


(i.e.,  sometimes,  somewhat,  some,  and  moderate  extent). 
Awareness  of  the  importance  of  managing  fatigue  at  work 
remained  high,  even  though  the  reduced  importance  rating 
approached  significance. 

All  Attitude  Assessment  items,  including  measures  of 
behavioral  intention,  were  examined  between  pre-training 
and  follow-up.  A  notable  difference  between  response 
pattern  categories,  awareness  of  importance,  self-efficacy, 
and  behavioral  intention,  is  that  all  ratings  for  behavioral 
intention  (Figure  5) ,  were  lower  and  below  the  mid-point  of 
the  scale,  except  the  item  asking  How  much  effort  do  you  put 
toward  avoiding  fatigue?  Overall,  no  statistically  significant 
differences  were  found;  although,  shifts  in  four  of  the  six 
behavioral  intention  items  tended  in  the  positive  direction. 
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□  Pre-Training  B  Follow-Up 


Awareness  of  Importance 

How  important  do  you  think  it  is  to  not  be 
latigued  while  on  duty  in  your  line  of  work? 

To  what  extent  do  you  feel  that  managing 
fatigue  is  important? 


How  committed  are  you  to  managing  fatigue? 

How  nwtivated  are  you  to  avoid  feeling 
fatigued?  (n=l4) 


Self-Efficacy 

To  what  extent  do  you  think  you  can 
successfully  manage  fatigue? 

How  confident  are  you  that  you  will  be  able  to 
apply  the  strategies  learned  in  training? 

To  what  extent  do  you  think  that  you  can 
apply  strategies  to  manage  fatigue  at  home? 

To  what  extent  do  you  think  that  you  can 
apply  strategies  to  manage  fatigue  on  the  job? 


Behavioral  Intention 


■Less 


More . > 


- - - - . 

3.7 

3.5 

3.6  1 

1  3.3  1 

3.3,1 

, _ 3^2J 

< — less 

More 

How  often  do  you  consciously  think  about 

2l5l 

ways  to  keep  from  feeling  fatigued? 

How  frequently  do  you  change  your  behavior 

2JT 

or  schedule  to  keep  from  feeling  fatigued? 

3.1  1 

How  much  planning  do  you  do  to  keep  from 

3.0 

feeling  fatigued? 

■XT 

How  much  effort  do  you  put  towards  avoiding 

3.2 

fatigue? 

How  often  do  you  talk  toothers  (co-wcKkers, 
friends,  family)  about  fatigue  management? 

(n  14) 

3.2 

2.1  1 

-  231 

How  often  do  you  think  about  what  you  can 

2.6  1 

do  to  manage  fatigue? 

2.9  1 

Figure  5.  Difference  in  attitudes  before  training  to  follow-up  (mean) 
(n=15) 


Transfer 

Transfer  refers  to  the  use  of  fatigue  countermeasures 
covered  in  training  in  the  home  and  work  environments. 
Transfer  assumes  that  the  environments  offer  support  for 
and  opportunities  to  employ  countermeasure  recommen¬ 
dations.  This  is  more  likely  the  case  at  home,  where  there 
is  more  individual  control,  than  at  work. 

The  Behavioral  Inventory  items  were  used  to  evaluate 
the  degree  of  training  transfer  by  comparing  the  self-rated 


frequencies  of  engaging  in  good  and  bad  habits^  listed  in 
the  inventory.  The  33  good  habits  were  categorized  as  sleep 
routine  (1 1),  health  and  fitness  (14),  and  work-life  (8).  There 
were  10  bad  habits  related  to  sleep,  nutrition,  and  exercise. 


®  Frequency  ratings  were  made  on  a  5-point  scale,  where  “  1  ”  represented 
Never,  “2”  represented “3”  represented  Sometimes,  “4”  represented 
Often,  and  “5”  K’pKsenmA  Always. 
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^  Sleep  Routine 

1 .  Use  pre-sleep  routine 

2.  Sleep  in  the  bedroom 

3.  Modify  sleep  environment 

4.  Ask  not  to  be  disturbed 

5.  Silence  phone 

6.  Block  out  noise  from  bedroom 

7.  Keep  bedroom  cool 

8.  Get  out  of  bed  when  can't  sleep 

9.  Have  consistent  bedtime 

10.  Get  sufficient  sleep  on  days  off 

11.  Take  naps 


Slipping 


Succeeding 


7,10 


T  E 


3,4 


'4^ 


1,5 

♦ 


Lagging 


Gaining 


Seldom  Sometimes  Often 

Follow-up 


Always 


I  I  Health  &  Fitness 

1.  Avoid  caffeine  hours  before  bed 

2.  Avoid  cigarettes  before  bed 

3.  Avoid  alcohol  hours  before  bed 

4.  Eat  nutritious  meals 

5.  Eat  multiple  meals  or  snacks 

6.  Avoid  large  meals 

7.  Take  vitamins 

8.  Drink  plenty  of  water 

9.  Exercise  regularly 

10.  Stretch  muscles 

11.  Take  walks 

12.  Use  strength  training  exercises 

13.  Use  cardio  exercises 

14.  Avoid  exercise  3  hours  before  bed 

^^Work-Life 

1.  Change  work  schedule 

2.  Use  sunglasses  going  home  in  AM 

3.  Increase  exposure  to  bright  light 

4.  Get  up  and  move  around 

5.  Take  breaks 

6.  Use  relaxation  techniques 

7.  Communicate  schedule  with 
family/fiiends 

8.  Socialize  with  others 


Slipping 


Succeeding 

2 

B - 


5,  14 

□ 


1 

D 


7, 9, 13 


□jnj4-|4-6.io.iL-L 


12 

o 


Lagging 


Gaining 


Never  Seldom  Sometimes  Often 

Follow-Up 


Always 


Slipping 


Succeeding 


_ a^L 

- 


Lagging 


Gaining 


Seldom  Sometimes  Often 

Follow-Up 


Always 


Figure  6.  Occurrence  of  good  habits  before  training  vs  follow-up  (median)  (n=14) 


The  results  plotted  in  Figures  6  and  7  provide  a  relative 
comparison  of  follow-up  to  pre-training  median'”  rat¬ 
ings  for  the  good  and  bad  habits,  respectively.  Each  plot 

Median  represents  the  midpoint  of  a  distribution.  In  this  instance, 
the  median  is  the  value  at  or  below  where  50%  rated  their  frequency  of 
engaging  in  the  specified  behavior. 


is  separated  into  quadrants  labeled  Succeeding,  Gaining, 
Slipping,  and  Lagging  that  indicate  which  behaviors  have 
and  have  not  transferred.  The  quadrants  were  based  on  the 
following  criteria: 

•  Succeeding,  the  behavior  consistently  occurred  more 
often  than  sometimes-. 


®  Bad  Habits  to  avoid 

Slipping  1 

Succeeding 

2,  5,6 

1 .  Skip  exercise  | 

- ^ 

2.  Exercise  rieht  before  bed  c 

1 

9,10 

3 .  Eat  fast  unhealthy  food  o 

- 1 

- •- - 

4.  Use  caffeine  '1  e 

5.  Use  other  stimulants  *-  e  ' 

o 

1,3^ 

6.  Use  tobacco  ^ 

7.  Use  alcohol  to  relax  before  bed  2  - 

- ^ 

1 - 

8.  Watch  TV  in  bed 

9.  Watch  the  clock  in  bed  %  - 

■ - 

— 

10.  Sleep  less  on  days  off  ^ 

Lagging 

Gaming 

I  1  1  1  1  1 

Never  Seldom  Sometimes  Often  Always 

Follow-up 

Figure/.  Avoidance  of  bad  habits  before  training  vs.  follow-up  (median)  (n=14) 


•  Gaining;,  the  behavior  did  not  occur  more  often  than 
sometimes  until  after  training; 

•  Slipping',  before  training  the  behavior  occurred  more 
often  than  sometimes  but  was  not  sustained  at  follow- up; 

•  Lagging,  the  behavior  consistently  occurred  less  often 
than  sometimes. 

Some  good  habits  were  already  established.  The  gain 
in  frequency  of  using  two  sleep  routine  countermeasures 
provides  some  evidence  of  a  training  impact  on  good  habits. 
Figure  7  shows  that,  for  the  most  part,  attendees  have  suc¬ 
ceeded  in  avoiding  behaviors  that  interfere  with  restful  sleep. 

A  subsequent  examination  was  made  of  the  average  per¬ 
cent  of  attendees  (n=l4)  reporting  that  they  frequently  (i.e., 
ratings  of  either  often  or  always)  use  the  countermeasures 
presented  in  training  and  avoid  the  pitfalls  of  bad  habits. 
Results  presented  in  Tables  4  thru  7  reveal  more  attendees 
were  able  to  overcome  bad  habits  than  acquire  good  habits. 
Good  habits  with  the  positive  differences  (e.g.,  modifying 
sleep  environment,  avoiding  caffeine  before  bed,  and  taking 
breaks)  for  some  were  easier  to  employ. 

Additional  items  measuring  transfer  asked  attendees  if 
they  had  changed  behaviors  at  home  or  work  for  purposes 
of  managing  fatigue  and,  if  so,  how.  More  than  twice  as 
many  reported  they  had  modified  behaviors  at  home  six 
weeks  after  training  (7  of  15,  47%)  compared  to  before 
training  (4  of  24 , 1 7%) .  This  was  not  the  case  at  work— fewer 


reported  modifying  behavior  at  follow-up  (3  of  15,  20%) 
compared  to  before  training  (6  of  24,  25%). 

Table  8  lists  attendees  reported  behavior  changes  to 
manage  fatigue  at  home  and  work.  More  fatigue  counter¬ 
measures  were  used  at  home  than  at  work  following  the 
training.  All  of  the  reported  changes  in  behavior  at  home 
were  countermeasures  recommended  during  training. 

Potential  for  Training  Impact  on  Self-reported  Fatigue 

Measures  ofself-reported  fatigue  collected  before  training 
and  six  weeks  following  are  summarized  in  Table  9.  No  dif¬ 
ferences  were  found  to  be  statistically  significant,  although 
attendees  tended  to  report  higher  rates  of  insufficient  sleep 
and  trouble  sleeping  at  follow-up.  Without  knowing  the 
details  of  their  daily  lives,  one  possible  explanation  for  the 
outcome  is  based  on  the  finding  that  the  training  heightened 
knowledge  of  sleep  and  fatigue  basics,  which  would  influ¬ 
ence  self-assessments  of  what  it  means  to  get  an  adequate 
amount  of  quality  sleep. 

Difference  in  total  scores  for  the  10-item  FAS  (n=12, 
M„  .  .  =21.6,  M„ ,,  =21.4)  between  the  two  data  col- 

rre-traimng  rollow-up 

lection  events  were  also  not  statistically  significant.  The 
range  of  possible  total  scores  is  10  to  50,  where  higher 
numbers  indicate  more  fatigue.  Figure  8  shows  the  average 
self-ratings  across  the  individual  FAS  items  that  contributed 
to  the  total  score. 
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Table  4.  Average  percent  (mean)  of  attendees  reporting  that  they  often  or  always 
used  good  Sleep  Routine  habits  recommended  in  training  _ 


Pre- 

training 

Follow¬ 

up 

Difference  I 

Modifying  sleep  environment 

14.3 

42.9 

284 

Asking  not  to  be  disturbed 

28.6 

50.0 

— 

]21.4 

Using  pre-sleep  routine 

14.3 

28.6 

— 

□  14 

■.3 

Silencing  phone  when  sleep 

28.6 

28.6 

0.0 

Blocking  out  noise  from  bedroom 

50.0 

50.0 

0.0 

Taking  naps 

14.3 

14.3 

0.0 

Using  bedroom  for  sleep 

50.0 

46.2* 

-3 

i.8  □ 

Getting  out  bed  when  can't  sleep 

21.4 

14.3 

-7.1 

Keeping  bedroom  cool 

57.1 

50.0 

-7.f 

□ 

Have  a  consistent  bedtime 

50.0 

35.7 

-14 

,.3  □ 

Getting  sufficient  sleep  on  days  off 

71.4 

42.9 

-|8.6 

_ 

Table  5.  Average  percent  (mean)  of  attendees  reporting  that  they  often  or  always 
used  good  Health  &  Fitness  habits  recommended  in  training 


Difference 


Pre- 

training 

Follow¬ 

up 

Avoiding  caffeine  before  bed 

50.0 

64.3 

Exercising  regularly 

21.4 

23.1* 

Doing  cardio  exercises 

21.4 

21.4 

Eating  nutritious  meals 

28.6 

28.6 

Avoiding  large  meals  before  bed 

28.6 

28.6 

Avoiding  cigarettes  before  bed 

57.1 

57.1 

Stretching  muscles 

28.6 

21.4 

Avoiding  alcohol  hours  before  bed 

42.9 

35.7 

Drinking  water 

50.0 

38.5* 

Taking  walks 

42.9 

28.6 

Eating  multiple  meals  or  snacks 

28.6 

14.3 

Avoiding  exercise  hours  before  bed 

46.2* 

30.8* 

Doing  strength  training 

23.1 

7.1* 

Taking  vitamins 

50.0 

21.4 

-14, 

-14, 

-15.4 

-15.l> 


128.6 


-7.1 

-7.1 


il.5 

,3  [ 

3  [ 


]  1.6 
0.0 
0.0 
0.0 
0.0 


L 


14.3 


n=13 
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Table  6.  Average  percent  (mean)  of  attendees  reporting  that  they  often  or  always 
used  good  Work-Life  habits  recommended  in  training 


Pre- 

training 

Follow¬ 

up 

Taking  breaks 

42.9 

50.0 

Socializing  with  others 

14.3 

21.4 

Using  relaxation  techniques 

0.0 

0.0 

Communicating  schedule  with 
family/friends 

21.4 

21.4 

Changing  work  schedule 

7.1 

0.0 

Using  sunglasses  when  going  home 
in  the  AM 

35.7 

28.6 

Getting  up  and  moving  around 

28.6 

14.3 

Increasing  exposure  to  bright  light 

21.4 

0.0 

Difference 


0.0 

0.0 


-7.1 


Table  7.  Average  percent  (mean)  of  attendees  reporting  that  they  often  or  always 
avoided  Bad  Habits  as  recommended  in  training 


Difference 


Pre¬ 

training 

Follow¬ 

up 

Not  watching  the  clock  in  bed 

57.1 

71.4 

Not  using  caffeine 

21.4 

35.7 

Not  exercising  right  before  bed 

78.6 

92.9 

Not  using  tobacco 

64.3 

* 

76.9 

Not  watching  TV  in  bed  before 
sleep 

50.0 

* 

61.5 

Not  skipping  exercise 

23.1 

* 

30.8 

Not  eating  unhealthy  fast  food 

35.7 

42.9 

Not  using  stimulants,  besides 
caffeine 

92.9 

92.9 

Not  using  alcohol  before  bed 

71.4 

71.4 

Not  sleeping  less  on  days  off 

64.3 

57.1 

I  14.3 
I  14.3 
I  14.3 
12.6 
11.5 


7.7 

7.2 


0.0 

0.0 


-7.1 


*n=13 
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Table  8.  Behaviors  used  to  manage  fatigue  (with  count)  at  home  and  work  before  and  following 
training 


At  Home 

At  Work 

Pre-Training 

(n=4  of  24) 

Follow-up 

(n=7  of  15) 

Pre-Training 

(n=6  of  24) 

Follow-up 

(n=3  of  15) 

Get  sufficient  sleep  (2) 

Get  sufficient  sleep  (2) 

Think  before  acting  (3) 

Stay  calm  (2) 

Limit  activities  before 
bedtime  (2) 

Don’t  watch  TV  in  bed 
(2) 

Change  hours  worked 
(1) 

Better  schedule  (1) 

Get  out  of  bed  if  can’t 
sleep  (1) 

Modify  sleep 
environment  (1) 

Eat  healthier  (1) 

Eat  light  meal  before 
bedtime  (1) 

Block  out  noise  from 
bedroom  (1) 

Walk,  instead  of  riding 
cart  (1) 

Stick  to  consistent 
bedtime  (1) 

Stick  to  consistent 
bedtime  (1) 

Drink  more  water  (1) 

Treatment  for  sleep 
disorder  (1) 

Communicate  with 
family  (1) 

Quiet  time  at  breaks  (1) 

Avoid  eating  late  (1) 

Light  snacks  at  breaks 
(1) 

Of  the  six  attendees  who  reported  previously  receiving  fatigue  training/education,  one  also  reported  changing 
behaviors  at  home  and  work  and  two  reported  changing  behaviors  at  work  to  manage  fatigue. 


Table  9.  Percent  of  self-ratings  indicating  potential  fatigue  (n=15) 

Pre-Training 

Follow-up 

Inadequate  amount  of  sleep  on  work  days 

78.6 

81.3 

Inadequate  amount  of  sleep  on  days  off 

40.0 

60.0 

‘Bothered  by  trouble  sleeping  in  last  30  days 

20.0 

40.0 

‘Bothered  by  tiredness  or  fatigue  in  last  30  days 

43.3 

33.4 

percent  responding  not  nearly  enough,  could  do  with  a  lot  more,  or  could  do  with  a  little  more 
^percent  responding  moderately,  very  much,  or  extremely 


Always 

Often 

About 

halftime 

Seldom 

Never 


Physically 

Bothered  by 

Get  tired 

Mentally 

Not  enough 

Problems 

Problems 

Don't  do 

No  desire 

Can't 

exhausted 

fatigue 

quickly 

exhausted 

energy 

starting 

things 

thinking 

clearly 

much 
during  the 
day 

to  do 
anything 

concentrate 
while  doing 

Figure  8.  Fatigue  Assessment  Scale  items  (mean)  (n=12) 
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Reaction  to  the  Training  Experience 

Instructional  qualities"  of  the  three-hour  classroom 
session  were  assessed  on  the  post-measure.  Results,  based 
on  33  attendees  who  provided  data  on  their  reaction  to 
the  training  experience,  were  overwhelmingly  positive.  All 
attendees  agreed  that  the  training  was  informative  and  use¬ 
ful.  The  vast  majority  agreed  that  training  was  interesting, 
worthwhile,  and  should  be  conducted  for  others  (97 %) ,  and 
that  the  material  was  well  organized  (96%)  and  presented 
at  the  appropriate  level  (94%). 

Training  Improvements 

Responses  to  three  open-ended  items  asking  for  most 
and  least  useful  topics  and  changes  to  improve  the  training 
formed  the  basis  of  feedback  on  needed  improvements.  Re¬ 
sults  from  a  content  analysis  conducted  on  attendee  feedback 
are  in  Table  10.  With  the  exception  of  a  few  topics  that 
require  review,  the  content  overwhelmingly  met  the  needs 
of  the  training  audience.  In  fact,  suggested  improvements 
were  requests  for  more  training  in  terms  of  lengthening 
the  course  and  including  more  training  materials/media. 

Other  needed  improvement  to  the  training  will  be  driven 
by  the  results  indicating  instability  in  changed  attitudes, 
lower  than  expected  self-efficacy  regarding  managing  one’s 
fatigue,  and  differences  between  rates  of  self-reported  use  of 
good  habits  and  avoidance  of  bad  habits,  which  indicate  that 
it  is  easier  to  discard  old  behaviors  than  acquire  new  ones. 

DISCUSSION 

The  evaluation  of  the  fatigue  countermeasures  training 
content  supports  findings  from  other  industries  that  fatigue 
countermeasures  training  can  bean  effective  tool  to  increase 
prevention  and  management  of  fatigue  at  the  individual  level. 
Even  though  the  training  content  was  under  development, 
it  increased  maintenance  employees’  awareness  of  fatigue 
risk  management  strategies  and  consequences  of  fatigue, 
which  influenced  some  to  change  their  daily  activities  in 
an  effort  to  manage  fatigue.  Effectiveness  of  the  training 
is  partly  attributable  to  the  customization  of  content  for 
AMTs  and  the  personalization  of  fatigue  health  risks  and 
benefits  of  fatigue  management. 

The  positive  shifts  in  the  following  are  notable  given  the 
resistance  of  attitudes  and  behaviors  to  change  (see  Erber, 
Hodges,  &  Wilson,  1 995;  Wood,  2000;  Ford  &  Ford,  2010): 
(a)  awareness  of  the  importance  of  not  being  fatigued  at 
work  and  commitment  to  manage  fatigue  increased  im¬ 
mediately  after  training,  and  (b)  modifications  of  sleep 
routines  reported  at  follow-up  (more  likely  to  sleep  in  the 
bedroom  and  block  out  noise  from  the  bedroom  at  follow- 

"  Instructional  qualities  were  assessed  on  a  5-point  Likert  scale,  with 
the  following  labels:  Strongly  Disagree,  Disagree,  Neutral,  Agree,  and 
Strongly  Agree. 


Table  10.  Percent  of  feedback  responses 
by  evaluation  item 

Most  Useful  Topics(32  comments  from 
25  attendees) 

% 

Strategies  for  better  sleep 

15.6 

-  All  information 

-  Countermeasures 

-  Fatigue  basies 

-  Cireadian  rhythm 

12.5 

Fatigue  vs.  intoxication  comparison 

9.4 

-  Fatigue  management 

-  Sleep  disorders 

-  Sleep  process 

-  Sleep  requirements  &  evaluation 

6.3 

Least  Useful  Topics  (2  comments  from 

2  attendees) 

% 

-  Fatigue  statistics 

-  Sleep  log 

50.0 

Suggested  Improvements(ll  comments 
from  10  attendees) 

% 

More  class  time 

45.5 

-  More  training  media 

-  Customize  for  company 

18.2 

-  Adjust  pace 

-  Separate  classes  for  non-hourly  and 
hourly  employees 

9.1 

14  attendees  indicated  all  topics  were  useful 


up).  However,  few  reported  changes  in  work  behaviors  at 
follow-up  anda  rebound  occurred  back  to  pre-training  levels 
in  employees’  commitment,  motivation,  and  self-efficacy 
toward  managing  fatigue. 

Modifying  and  sustaining  new  behaviors  and  attitudes 
toward  fatigue  can  be  challenging,  since  lifestyle  changes 
are  required.  When  resistance  exists,  a  positive  approach 
would  be  to  treat  it  as  valuable  feedback.  For  instance, 
during  class  some  attendees  voiced  resistance  to  changing 
their  sleep  habits.  The  interaction  provided  the  instructor  an 
opportunity  to  generate  a  better  understanding  of  how  and 
why  they  needed  to  modify  their  sleep  environments.  An 
organization  can  also  productively  engage  resistance  when 
developing  or  revising  policies  and  procedures  to  reduce 
fatigue-related  safety  risks  by:  (a)  involving  employees,  (b) 
focusing  on  the  purpose  of  the  change,  (c)  resolving  past 
issues,  clarifying  objectives  and  strategies,  and  (d)  improving 
the  implementation  plan  (Ford  &  Ford,  2010).  Productive 
engagement  from  the  organization  requires  commitment 
to  reduce  fatigue  risk  on  the  job,  otherwise  employees’  old 
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patterns  of  behaviors  will  persist.  Lack  of  productive  engage¬ 
ment  from  the  organization  may  contribute  to  maintenance 
employees  feeling  powerless  to  manage  fatigue  at  work 
due  to  limited  or  no  control  over  the  work  environment, 
the  work  schedule,  and  the  accelerated  pace  and  elevated 
pressure  to  perform. 

Promoting  personal  and  public  safety  by  prescribing 
duty  limitations  and  rest  requirements  that  contain  the 
right  balance  of  safety,  science,  and  operational  efficiency 
for  aviation  maintenance,  equivalent  to  flight  crews,  re¬ 
mains  a  challenge.  Like  pilots  and  flight  attendants,  some 
AMTs  commute  across  time  zones  and  perform  consecutive 
extended  shifts  before  returning  home.  Because  there  are 
no  duty  hour  limitations  or  look-back  rest  requirements,'^ 
maintenance  personnel  appear  to  be  susceptible  to  high 
levels  of  physical  and  cognitive  fatigue.  Implementing  an 
effective  fatigue  countermeasures  training  program  is  just 
one  step  that  aviation  maintenance  organizations  should 
take  in  reducing  fatigue  risk  to  improve  public  safety.  An 
online  version  of  the  fatigue  countermeasure  training  for 
AMTs  (MXfatigue.com)  developed  by  the  FAA  is  available 
on-demand  and  at  no  cost  to  individuals  and  organizations. 
To  realize  the  safety  and  productivity  benefits  of  reduced 
fatigue  risk,  organizations  will  need  to  establish  fatigue 
management  policies  and  procedures,  such  as  a  Fatigue  Risk 
Management  System,  that  apply  the  concepts  and  reinforce 
attitudes  and  behaviors  taught  in  training. 

The  comprehensive  evaluation  approach  used  in  this 
study  is  rare,  as  training  evaluation  efforts  are  often  focused 
solely  on  attendee  reactions  to  the  instruction.  Evaluation 
results  were  for  purposes  of  improving  training  content 
and  not  intended  to  generalize.  Future  evaluations  of  the 
online  content  would  benefit  from  use  of  a  control  group 
that  does  not  receive  training;  random  assignment  of  par¬ 
ticipants  to  either  the  training  or  control  group;  inclusion 
of  employees  representing  all  work  schedules,  especially, 
alternative  work  schedules  (e.g.,  night  shifts  or  extended 
shifts)  that  can  render  AMTs  more  vulnerable  to  human 
fatigue;  reduce  questionnaire  length;  and  include  participa¬ 
tion  from  a  range  of  aviation  maintenance  organizations. 


Before  accepting  a  flight  assignment,  a  flight  crewmember  must  be 
able  to  prospectively  determine  that  he/ she  will  be  able  to  look  back  24 
hours  from  the  scheduled  completion  of  each  flight  segment  and  find 
a  legally  scheduled  rest  within  the  previous  24  hours. 
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